With the growing popularity of “cloud computing,” the number of computers needed to support cloud computing is increasing rapidly. A provider of a cloud infrastructure (“a cloud provider) typically organizes the computers into geographically distributed data centers that can each have thousands of computers. The data centers are distributed for various reasons. One reason is to reduce response time to clients by having clients connect to the closest data center. For example, clients in Europe may connect to a data center located in Europe, and clients in North America may connect to a data center in the United States. Another reason is to provide a backup in case of disaster. For example, if the data center in Europe was down because of a natural disaster, the clients in Europe could be connected to the data center in the United States. Although the response time may be somewhat slower because of the distance and because of the increased load on the data center in the United State, the clients can still have useful work done.
Because the data centers are geographically distributed, large amounts of data may need to be transferred between datacenters. For example, data of a data center may need to be periodically backed up to another data center. If the data center was to go down, then the other data center would have a current or near-current copy of the data. Although it may take longer to access the backed up data, it is still accessible. As another example, as the amount of stored data of a data center nears the storage capacity of the data center, certain stored data may need to be moved to another data center that has more storage capacity.
The amount of data that is moved between data centers can be well into the petabytes and approach an exabyte. The speed of moving of such large amounts of data between data centers is limited by the wide-area network (“WAN”) that connects the data centers. As a result, the movement of high-priority data (or high-priority traffic) can be delayed because of the movement of large amounts of relatively low-priority data (e.g., a large database that is being backed up). Because a data center can have tens of thousands of computers used by thousands of customers, it can be difficult to predict how much data the computers will attempt to move between the data centers at any given time. Many of the computers of different customers may attempt to move large amounts of data from one data center to another data center at the same time. As a result, the data center may not be able to provide the agreed-upon level of service to each customer. To help ensure that certain data movement traffic is not squeezed out by another data movement traffic or that customers receive the agreed-upon level of service, some cloud providers may dedicate certain amounts of bandwidth to different types of traffic. For example, a cloud provider may dedicate 70% of the bandwidth to one type of traffic and 30% to another type of traffic. Because of the unpredictable nature of data movement by customers and to some extent by the data center itself, a cloud provider may not always be able to provide the agreed-upon level of service to its customers.